Psychology
230 – Stats/Methods I
Research
Strategies and Validity (G&F Ch. 6)
I. Research Strategies
A. Descriptive Strategy
Focus is on describing (as opposed to explaining or predicting)
one variable at a time.
Descriptive
studies are numerically descriptive.
B. Correlational Strategy
Measuring two different variables for each participant in order to describe the relationship between the two variables.
The variables are usually measured as they occur naturally—without any attempt to manipulate or control them.
C. Experimental Strategy
The experimenter manipulates at least one IV. A DV is used to measure the effects of the IV.
4
characteristics of true experiments:
1 – MANIPULATION
2 – MEASUREMENT
3 –COMPARISON
4 – CONTROL
The Basic Components of an Experimental Research Study
Aim to establish a tentative cause and effect relationship between two variables but cannot satisfy all of the strict requirements needed for a true experiment (often cannot not meet all of the above requirements in a natural settings)
There IS a manipulation, but control is lacking
Uses some of the rigor and control used in true experiments. But in some way, lack the control found in true experiments (usually lack random assignment of participants to conditions), so ability to draw a causal inference is impaired
E. Non-Experimental Strategy
Two different groups are involved, but the researcher does NOT manipulate a variable to create the groups
No control is exerted over extraneous variables. No causal explanation can be obtained. Only able to determine if there is a relationship between the two groups (same, different).
How is a non-experimental study different from a correlation study?
Q: Consider the following hypothesis: Stress at work is
related to family conflict at home.
(a)
How might you investigate this
hypothesis using the correlational strategy?
(b)
How might you investigate this
hypothesis using the nonexperimental strategy?
(c)
How might you investigate this
hypothesis using the experimental strategy?
II. Validity
How well does the study answer the question it was intended to answer?
**A researcher's ultimate goal: “This is what happened, and this is what it means.” Any factor that raises doubts about the research results or about the interpretation of the results is a THREAT TO VALIDITY.
Extent to which other causes are ruled out—extent to which the difference in behavior can unambiguously be attributed to the manipulation
B. External Validity
Extent to which we can generalize the results of a research study to people, settings, times, measures, and characteristics other than those used in that study.
A cold, hard fact:
It is impossible for a study to totally eliminate all threats to validity. Each study is a set of decisions and compromises. Often there's a trade-off between internal and external validity. Therefore, a single study can never "prove" anything on its own. We need to get converging evidence. Also, we must always be critical consumers of research and make our own decisions about validity and quality of the research.
Q: A researcher conducts a study with 6-year-old children at a summer computer
camp for gifted children. However, the researcher suspects that different
results would be obtained if the study were conducted with nongifted 6-year-old
children. Does this study have a problem with internal validity or external
validity?
Q: A researcher finds that
college students are more anxious near final exams in December than at the
beginning of the semester in September. However, it is not clear whether the
anxiety is caused by exams or by the change in season. Does this study have a
problem with internal validity or external validity?
III. Threats to Validity
A. Threats to Internal Validity
Extraneous variables vs. Confounding variables
Extraneous variable--any variable in a study other than the two variables of
interest
Confounding variable--an extraneous variable (usually unmonitored) that is
allowed to change systematically along with the two variables being
studied. Muddles the results...Prevents you from establishing a causal link between your two
variables of interest.
A. Threats to Internal Validity
Category 1: For all studies:
Environmental variables
Category 2: For studies comparing different groups:
assignment bias
Category 3: For studies comparing one group over time:
history effects--environmental events other than the treatment that occur between the first treatment condition and the last treatment condition and may affect the results
maturation--systematic changes in physiology or psychology that occur during study and may affect the participants' scores
instrumentation--changes in a measuring instrument that take place during the course of the study
testing effects--practice effects, fatigue, carry-over effects
regression toward the mean
B. Threats to External Validity
Category 1: Generalizing across participants
1. Subject selection bias (cost-restrictive sampling)
2. College students
3. Volunteer bias
4. Participant characteristics
5. Cross-species generalizations
1. Novelty effect
2. Reactivity
3. Demand characteristics
4. Multiple treatment interference
Category 3: Generalizing across experiments
1. Experiment bias
2. Experimenter characteristics
Category 4: Generalizing across features of the measures
1. Sensitization
2. Generality across response measures
3. Time of measurement